1. Field of the Invention
The invention relates to automatic suggesting or processing of enhancements of a digital image using information gained from identifying and analyzing faces appearing within the image, and in particular method of detection the image orientation using face detection. The invention provides automated orientation detection for photographs taken and/or images detected, acquired or captured in digital form or converted to digital form, by using information about the faces in the photographs and/or images.
2. Description of the Related Art
Viola and Jones in the paper entitled “Robust Real Time Object Detection” as presented in the 2nd international workshop on Statistical and Computational theories of Vision, in Vancouver, Canada, Jul. 31, 2001, describe a visual object detection framework that is capable of processing images extremely rapidly while achieving high detection rates. The paper demonstrates this framework by the task of face detection. The technique is based on a learning technique where a small number of critical visual features yield a set of classifiers.
Yang et al., IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, No. 1, pages 34-58, give a useful and comprehensive review of face detection techniques January 2002. These authors discuss various methods of face detection which may be divided into four main categories: (i) knowledge-based methods; (ii) feature-invariant approaches, including the identification of facial features, texture and skin color; (iii) template matching methods, both fixed and deformable and (iv) appearance based methods, including eigenface techniques, statistical distribution based methods and neural network approaches. They also discuss a number of the main applications for face detections technology. It is recognized in the present invention that none of the prior art describes or suggests using detection and knowledge of faces in images to create and/or use tools for the enhancement or correction of the images according to the invention as set forth in the claims below, nor as described in detail below as preferred and alternative embodiments.
Balluja, 1997 describes methods of extending the upright, frontal template based face detection system to efficiently handle all in plane rotations, this achieving a rotation invariant face detection system.
a. Faces as Subject Matter
It is well known that human faces are the most photographed subject matter for the amateur and professional photographer. Thus it is possible to assume a high starting percentage for algorithms based on the existence of faces in them.
b. Orientation
The camera is usually held horizontally or vertically, in counter clockwise or clockwise in relations to the horizontal position when the picture is taken, creating what is referred to as a landscape mode or portrait mode, respectively. Thus most images are taken in either one of the three orientations, namely landscape, clockwise portrait and counterclockwise portrait. When viewing images, it is preferable to determine ahead of time the orientation of the camera at acquisition, thus eliminating a step of rotating the image and automatically orienting the image. The system may try to determine if the image was shot horizontally, which is also referred to as landscape format, where the width is larger than the height of an image, or vertically, also referred to as portrait mode, where the height of the image is larger than the width. Techniques may be used to determine an orientation of an image. Primarily these techniques include either recording the camera orientation at an acquisition time using an in camera mechanical indicator or attempting to analyze image content post-acquisition. In-camera methods, although providing precision, use additional hardware and sometimes movable hardware components which can increase the price of the camera and add a potential maintenance challenge. However, post-acquisition analysis may not generally provide sufficient precision. Knowledge of location, size and orientation of faces in a photograph, a computerized system can offer powerful automatic tools to enhance and correct such images or to provide options for enhancing and correcting images.
c. Face Recognition as a Function of Orientation
It is a well known fact for one familiar in the art of face recognition that the human visual system is very sensitive to the orientation of the faces. As a matter of fact, experiments indicated that the way the human mind stores faces is different for upright and inverted faces, as described in Endo, 1982. In particular, recognition of inverted faces is known to be a difficult perceptual task. While the human visual system performs well in recognizing different faces, performing the same task with inverted faces is significantly worse. Such results are illustrated for example in Moses, 1994, where face memory and face recognition is determined to be highly orientation dependent. A detailed review of face recognition of inverted faces is available in Valentine, 1988.
It is therefore only natural that artificial intelligence detection algorithms based on face related classifiers may have the same features of being orientation variant.
d. Image Classifiers for Scene Analysis:
Even though human beings have no problem to interpret images semantically, the challenge to do so using artificial intelligence is not that straight forward. A few methods are available to those familiar in the art of image and pattern recognition that separate images using a learning based descriptor space. Such methods are using a training set and a maximization methods of likelihood. Examples of such methods includes the Adatron (1989) method as described by Analauf et. al incorporated herein by reference. Other work includes scene analysis such as the work by Le Saux Bertrand et al (2004).